Imaging apparatus capable of switching display methods

ABSTRACT

An imaging apparatus comprises an image pickup unit, a cutout image generation unit for cutting out a specified area in a pickup image taken by the image pickup unit to generate a cutout image enlarged at a specified magnification, an image display unit for displaying one or both of the pickup image taken by the image pickup unit and the cutout image generated by the cutout image generation unit, a display image control unit for controlling a method of displaying an image the image display unit displays, a manual focus operation unit for the user to control through manual operation the focus position of the image pickup unit, and a manual zoom operation unit for the user to control the zoom magnification of the image pickup unit.

INCORPORATION BY REFERENCE

This application is a continuation application of application Ser. No.14/093,089 filed Nov. 29, 2013, which is a continuation of applicationSer. No. 12/331,501, filed on Dec. 10, 2008, now U.S. Pat. No.8,599,244, with an issue date of Dec. 3, 2013, which claims the benefitof Japanese Application No. JP 2007-321486 filed on Dec. 13, 2007, inthe Japanese Patent Office, the disclosure of which are incorporatedherein by reference.

BACKGROUND OF THE INVENTION

The present invention relates to an imaging apparatus with an imagepickup unit and an image display unit.

As a background art in the present technical field, one may refer toJP-A-11-341331, for example. The Patent Document aims at “providing anelectronic imaging apparatus capable of performing manual focus controlspeedily and steadily with simplified construction” and discloses asolving technique reading “in an electronic imaging apparatus capable ofswitching between manual focus (including PF) and auto-focus, a systemfocus controller 80 controls an electronic zoom circuit 61 during manualfocus so that a moving picture of a subject enlarged at a predeterminedmagnification higher than that during auto-focus may be displayed on aliquid crystal monitor 63. Thus, the enlarged image or video can bedisplayed on the liquid crystal monitor 63 during manual focus, ensuringthat visual recognition of correctness of in-focus state can befacilitated to permit a manual focus adjustment to be made speedily andsteadily. And besides, since an enlarged moving picture can bedisplayed, an in-focus operation can be fulfilled exactly in respect ofa moving picture.”

SUMMARY OF THE INVENTION

In recent digital video camera and still camera, a liquid crystaldisplay is generally mounted which is used to confirm a subject and anangle of field or view during image pickup. But, as the number of pixelsof an image-pickup device increases progressively in compliance with,for example, high definition (High Vision), the number of pixels becomessmall with the pixel pitch rendered decreased in a small-size liquidcrystal display, giving rise to a problem that details of a subject tobe desired for practically photographing is hardly discernible and exactin-focus and exposure states can hardly be confirmed, making itdifficult to make adjustments manually.

In the aforementioned JP-A-11-341331, by enlarging a moving picture of asubject at a predetermined magnification and displaying it on the liquidcrystal monitor during manual focusing, the user can be allowed tooperate manual focus while watching details of the subject. But when,even in the course of manual focusing, the zoom magnification, forexample, is desired to be changed, visibility of the entire angle offield is more convenient for operation than watching details of asubject, demonstrating that an image display complying with the contentsof operation by the user can promote easy-to-operate capability.

For example, the user's convenience can conceivably be ameliorated bysupporting the user in watching more easily a liquid crystal displaycommensurate with the use purpose either through using the interface foroperation of various camera functions in combination with control of amethod for displaying an image on the liquid crystal display to enablethe user to watch more easily a liquid crystal display meeting usepurpose or through combining with the result of image recognition toautomatically control the method for displaying an image on the liquidcrystal display but, for example, the aforementioned JP-A-11-341331fails to consider this point.

To eliminate the disadvantages encountered in the background art, thepresent invention has its object to improve the user's convenience inhandling an imaging apparatus and especially, to improve the conveniencefor the user to use various functions while watching the displaymonitor.

As an example, the above object can be accomplished by the presentinvention through inventions as recited in the scope of claims.

Other objects, features and advantages of the invention will becomeapparent from the following description of the embodiments of theinvention taken in conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a first schematic diagram showing an imaging apparatusaccording to a first embodiment of the present invention.

FIG. 2 is a diagram illustrating an example of an image display methodin the first embodiment of the invention.

FIG. 3 is a first flowchart showing an example of the control sequencein the image display method in the first embodiment of the invention.

FIG. 4 is a second schematic diagram showing the imaging apparatus inthe first embodiment of the invention.

FIG. 5 is a second flow chart showing an example of the control sequencein the image display method in the first embodiment of the invention.

FIG. 6 is a third schematic diagram showing the imaging apparatus in thefirst embodiment of the invention.

FIG. 7 is a diagram illustrating an example of control of switching theimage display method in the first embodiment of the invention.

FIG. 8 is a schematic diagram showing an imaging apparatus according toa second embodiment of the invention.

FIG. 9 is a diagram illustrating an example of a method of controlling acutoff area of a cutoff image in the second embodiment of the invention.

FIG. 10 is a diagram illustrating an example of a method of controllingan image display method in the second embodiment of the invention.

FIG. 11 is a schematic diagram showing an imaging apparatus according toa third embodiment of the invention.

FIG. 12 is a diagram illustrating an example of an image display methodin the third embodiment of the invention.

DESCRIPTION OF THE INVENTION

Preferred embodiments of this invention will now be described withreference to the accompanying drawings.

Embodiment 1

Referring first to FIG. 1, there is illustrated a first schematicdiagram showing an imaging apparatus according to a first embodiment ofthe present invention. The imaging apparatus shown in FIG. 1 comprisesan image pickup unit 0101, a cutout image generation unit 0102, an imagedisplay unit 0103, a display image control unit 0104, a manual focusoperation unit 0105 and a manual zoom operation unit 0106.

In the imaging apparatus as shown in FIG. 1, the image pickup unit 0101,in which zoom and focus are variable, includes a group of lensesrepresenting a zoom lens and a focus lens, a shutter, an image-pickupdevice such as CCD or CMOS, an AGC, an AD, a camera signal processingDSP, a moving picture processing LSI and a timing generator and operatesto perform image pickup based on photoelectric conversion and deliverimage or video data and image or video data for display which is reducedin accordance with the resolution of a display area of the image displayunit 0103. The image data for display may be generated by thinning theoriginal image data at intervals of a predetermined number of pixels orby performing an image reduction process based on a filter operationusing the moving picture processing LSI. The image data can be outputtedto image recording unit, not shown, so that a moving picture and a stillpicture may be recorded in a recording medium such as DVD, BD or HDD oralternatively, can be outputted to a TV monitor or PC display, notshown, so as to be displayed thereon. The cutout image generation unit0102 selects a predetermined area of the image data delivered out of theimage pickup unit 0101 and processes an image within the area atcoincident or ×1 magnification or at a predetermined enlargementmagnification so as to deliver the resultant image as cutout image data.The ×1 magnification herein signifies that the resolution of a picked-upimage equals that of an output image to permit delivery dot by dot. Theimage display unit 0103 is a monitor unit such as a liquid crystaldisplay attached to the imaging apparatus and it responds to informationconcerning control of an image display method outputted from the displayimage control unit 0104 to acquire and display the image data fordisplay delivered out of the image pickup unit 0101 or the cutout imagedata delivered out of the cutout image generation unit 0102. An exampleof the image display method will be described later with reference toFIG. 2. The display image control unit 0104 acquires from the manualfocus operation unit 0105 and manual zoom operation unit. 0106information concerning focus operation and zoom operation to determinean image display method of the image display unit 0103 on the basis ofthe captured information and outputs information for controlling theimage display method to the image display unit 0103. A method ofdetermining the image display method will be described later withreference to FIG. 3. The manual focus operation unit 0105 having a focusposition input interface such as joint dial, right/left button or leveroperates an amount of control of the focus position on the basis ofinput information from the focus position input interface and controlsthe position of the focus lens of the image pickup unit 0101 on thebasis of the control amount. This allows the user to control the focusposition manually. The manual focus operation unit 0105 also outputs tothe display image control unit 0104 information as to whether the focusoperation is in progress. The manual zoom operation unit 0106 having azoom magnification input interface such as joint dial, right/left buttonor lever operates an amount of control of the focus position on thebasis of input information from the zoom magnification input interfaceand controls the position of the zoom lens on the basis of the controlamount. The manual zoom operation unit 0106 also outputs to the displayimage control unit 0104 information as to whether the zoom operation isin progress. The operation of information for control of the imagedisplay method the display image control unit 0104 performs, theoperation of control amount of the focus position the manual focusoperation unit 0105 performs and the operation of control amount of thezoom magnification the manual zoom operation unit 0106 performs aretypically executed with the help of a microcomputer, camera signalprocessing DSP and dedicated LSI incorporated in the camera.

Turning to FIG. 2, an example of an image display method in the firstembodiment of the invention is illustrated. For an image display shownin FIG. 2, control of the display method is executed by means of thedisplay image control unit 0104 and an image display is executed bymeans of the image display unit 0103. In FIG. 2, an ordinary or normalpickup image is displayed on the liquid crystal display as shown at (a),a cutout image is displayed on the liquid crystal display as shown at(b), juxtaposition of the normal pickup image and the cutout image whichcorrespond to horizontal halves of a display area, respectively, isdisplayed on the liquid crystal display as shown at (c) andsuperimposition of the normal pickup image and the cutout image whichcorrespond to two divisions of a small area and the other area of thedisplay area, respectively, is displayed on the liquid crystal displayas shown at (d). The normal pickup image displayed on the image displayunit 0103 as mentioned herein corresponds to reproduction of image datafor display which is delivered out of the image pickup unit 0101 whilematching with the resolution of the display area of image display unit0103 and the cutout image displayed on the image display unit 0103corresponds to reproduction of cutout image data which is delivered outof the cutout image generation unit 0102. It is to be noted that at (a)in FIG. 2, dotted line does not indicate a subject image displayed on anactual image but indicates a cutout area of cutout image for convenienceof explanation and the image at (b) in FIG. 2 corresponds to a displayat ×1 magnification of the area surrounded by dotted line at (a) in FIG.2. In this manner, the image is displayed on the liquid crystal displayat the same resolution as that of the actually picked up image andtherefore, the displayed image is not adversely affected by degradationin picture quality attributable to an image reduction process, thusimproving the visual recognition capability. Thanks to this advantage,when the user takes a picture by confirming an image with the help ofthe liquid crystal display, the auto-focus and auto-exposure conditionsnear a noticed objective can be confirmed and manual focus operation andexposure operation can be executed more accurately. In this example, thecutout image is displayed at ×1 magnification but the pixel pitch of theliquid crystal display is extremely smaller than that of the normal TVmonitor or PC display and hence the display may be enlarged at apredetermined magnification. At that time, if an integer-timesenlargement process, for example, is carried out, the picture qualitydegradation attributable to the interpolation process can be suppressedas far as possible. For the image cutout area determination and theenlargement magnification in the course of the cutout image generation,setting values according to some modes may be set and written in anEEPROM at the time of, for example, shipment to enable the user to inputa selected value through an input interface, not shown, while watchingthe menu screen or an image cutout area may be set freely by the userthrough the use of a joint dial or a touch panel not shown. In thismanner, the user can be allowed to access a display of a noticed areaeasily at ×1 magnification or at an enlargement magnification to promotethe visual recognition capability. Further, by displaying the normalpickup image and the cutout image simultaneously as shown at (c) and (d)in FIG. 2, the user can watch the entire angle of field and details ofthe subject image at a time. Thus, upon taking a picture, confirmationof the entire composition and confirmation of focus and exposure nearthe subject can be carried out concurrently. Advantageously, at thattime, with the two image displayed in a juxtaposed relationship as shownat (c) in FIG. 2, the two images freed from superimposition can beconfirmed and with the two images displayed in a superimposedrelationship as shown at (d) in FIG. 2, the two images keeping theiraspect ratios unchanged can be confirmed. It will be noticed that inrespective drawings in FIG. 2, a letter “MF” or “MF×1” is indicated atthe upper left corner of the display image. Then, MF signifies that thepresent imaging apparatus performs the focus control in the manual focusmode and ×1 signifies that the image is displayed at the coincidentmagnification. In this manner, by providing a superimposed display ofletter information indicative of the state of each function of theimaging apparatus and the image enlargement magnification of the cutoutimage, the user can conveniently confirm the present use condition at aglance.

Reference will now be made to FIG. 3 showing an example of the sequenceof controlling an image display method in the first embodiment of theinvention. The control sequence for the image display method as shown inFIG. 3 is executed by means of the display image control unit 0104.

In the image display method control sequence of FIG. 3, in ST0301, theimage display unit 0103 is so controlled as to display a normal pickupimage after start of the imaging apparatus. In ST0302, operationinformation as to whether the user operates manual focus is capturedfrom the manual focus operation unit and the program branches to ST0303if the operation is detected but to ST0304 if the operation is notdetected. In the ST0303, the image display unit 0103 is so controlled asto display a cutout image. In the ST0304, operation information as towhether the user operates manual zoom is captured from the manual zoomoperation unit and the program branches to ST0305 if the operation isdetected but to ST0306 if the operation is not detected. In the ST0305,the image display unit 0103 is so controlled as to display the normalpickup image. In the ST0306, it is decided whether or not there occurs acondition of ending the display on the liquid crystal display by turningoff the imaging apparatus through a power supply switch not shown and inthe presence of liquid crystal display ending condition, the sequence isended but if not, the program repeats itself from the ST0303. As will beseen from the above, the image display method is automaticallycontrolled differently for the manual focus operation and the manualzoom operation, so that the user can conduct an exact focus operation bywatching the cutout image during the manual focus operation whereas theuser can determine the composition by confirming the entire angle ofview while watching the normal pickup image during the manual zoomoperation, thereby making it possible to perform an image displaycomplying with the user's use condition. In the present sequence thezoom operation is decided after completion of the focus operationdecision but this is a mere example and obviously, the order can beinverted or the two types of decision can be made at a time. Further,the synthesized image as shown at (c) or (d) in FIG. 2 may substitutefor the cutout image so as to provide the image to be displayed duringthe manual focus operation and besides, the user may select in advancemethods of displaying images during the manual focus operation andmanual zoom operation through a menu screen, for example, thus beingallowed to perform an operation while watching a preferable image.

Illustrated in FIG. 4 is a second schematic diagram showing the imagingapparatus in the first embodiment of the invention. There are seen inFIG. 4 the image pickup unit 0101, the cutout image generation unit0102, the image display unit 0103, the display image control unit 0104,an image recording unit 0407 and an image record operating unit 0408.

In the imaging apparatus as shown in FIG. 4, the image pickup unit 0101,cutout image generation unit 0102 and image display unit 0103 operatesimilarly to those in the imaging apparatus exemplified in FIG. 1. Thedisplay image control unit 0104 acquires information concerning imagerecording operation from the image record operating unit 0408 and on thebasis of the captured information, determines an image display method ofthe image display unit 0103 to output control information for the methodto the image display unit 0103. A method for determination of the imagedisplay method will be described later with reference to FIG. 5. Theimage recording unit 0407 includes a moving picture processing LSI andapplies a moving picture compression process to image data delivered outof the image pickup unit 0101 and records compressed image data on arecording medium such as DVD, BD or HDD. The record operating unit 0408includes a record operation input interface such as push button and onthe basis of input information from the record operation inputinterface, controls timings of causing the image recording unit 0407 tostart, pause and stop. The record operating unit 0408 also delivers tothe display image control unit 0104 information as to whether therecording operation is in progress. The timings of the recording therecord operating unit 0408 conducts are typically controlled by themicrocomputer or the camera signal processing DSP incorporated in thecamera.

A second flowchart of another example of the sequence of controlling theimage display method in the first embodiment of the invention is shownin FIG. 5. The control sequence for the image display method of FIG. 5is executed by the display image control unit 0104.

Referring to the sequence of controlling the image display method, inST0501, operation information as to whether a user operation forstarting image recording is present is acquired from the recordoperating unit and the program proceeds to ST0502 if the operation isdetected but if not, the process ends. In the ST0502, the image displayunit 0103 is so controlled as to display a normal pickup imageirrespective of the image display method at present. This is because inconsideration of the fact that the user manually controls the focus andexposure infrequently in the course of taking a picture actually, thedisplay method is switched automatically to the normal pickup image attiming of start of recording even when the image display unit 0103 isdisplaying a cutout image, with the result that the user can confirm theentire angle of view, being permitted to conduct an easy-to-handleoperation. In connection with the image to be displayed upon recording,the synthesized image as shown at (c) or (d) in FIG. 2 may substitutefor the normal pickup image, thus ensuring that for the sake ofimproving the operability, the user can be allowed to select in advancethrough the menu screen, for example, the method of displaying an imageto be displayed and the determination as to whether the present controlis to proceed during image recording.

A third schematic diagram of the imaging apparatus in the firstembodiment of the invention will now be described by making reference toFIG. 6. Illustrated in FIG. 6 are the image pickup unit 0101, the cutoutimage generation unit 0102, the image display unit 0103, the displayimage control unit 0104, the manual focus operation unit 0105, anauto-focus control unit 0609 and a focus control selecting unit 0610.

In the imaging apparatus as shown in FIG. 6, the image pickup unit 0101,cutout image generation unit 0102 and image display unit 0103 and manualfocus operation unit 0105 operate similarly to those in the imagingapparatus exemplified in FIG. 1. The display image control unit 0104acquires information concerning focus control selection operation fromthe focus control selection operation unit 0610 and on the basis of theacquired information, determines an image display method of the imagedisplay unit 0103 to output control information for the method to theimage display unit 0103. A method for determination of the image displaymethod and its display contents will be described later with referenceto FIG. 7. On the basis of image data delivered out of the image pickupunit 0101, the auto-focus control unit 0609 operates a focus positionsuitable for in-focus and on the basis of the operation result, controlsthe position of the focus lens of image pickup unit 0101. The focuscontrol selection operation unit 0610 having a focus control selectionoperation input interface such as a push button performs switching, onthe basis of input information from the focus control selectionoperation input interface, as to which one of the manual focus operationunit 0105 and the auto-focus control unit 0609 performs the control ofthe focus position in the image pickup unit 0101. The focus controlselection operation unit 0610 delivers the input information of thefocus control selection operation input interface also to the displayimage control unit 0104. The in-focus focal position operating processthe auto-focus control unit 0609 conducts is typically executed by themicrocomputer or the camera signal processing DSP incorporated in thecamera.

An example of the control of switching the image display method in thefirst embodiment of the invention is illustrated in FIG. 7. In FIG. 7, anormal pickup image is displayed on the liquid crystal display as shownat (a) when the focus control method is for auto-focus and the imagedisplay method is for normal image, a cutout image is displayed on theliquid crystal display as shown at (b) when the focus control method isfor auto-focus and the image display method is for cutout image, anormal pickup image is displayed on the liquid crystal display as shownat (c) when the focus control method is for manual focus and the imagedisplay method is for normal image and a cutout image is displayed onthe liquid crystal display as shown at (d) when the focus control methodis for manual focus and the image display method is for cutout image.Each time that the focus control selection operating unit 0610 receivesone input, the display image control unit 0104 switches the focuscontrol method and the image display method in order of sections (a),(b), (c) and (d) in FIG. 7. Thus, the user can perform switching betweenthe auto-focus and the manual focus and switching between the normalpickup image display and the cutout image display by simply recognizinghow many times the user depresses the push button, for example, andimprovements in operability can be realized. Operation by the pushbutton is exemplified herein but the control may be achieved byoperating, for example, a rotary dial or a touch panel.

The imaging apparatus according to the first embodiment has been setforth by way of three types of constructions for clear and betterunderstanding of the individual advantages the respective types have andobviously, a single imaging apparatus of the invention can be providedwith all of three types of constructions and functions. In such a case,all modes of control of the image display method including switchover ofthe image display method between the manual focus operation and themanual zoom operation, switchover thereof at the timing of startingrecord and switching the image display method through operation of thepush button may be rendered effective or alternatively, the user mayselect a necessary control mode from the menu screen.

In the present construction, the image display unit 0103 displays one ofthe pickup image and the cutout image delivered out of the image pickupunit 0101 and cutout image generation unit 0102, respectively, or asynthesis image of both the images but when the imaging apparatus has aframe memory adapted to store one screen of image data picked up by theimage pickup unit 0101 and the image display unit 0103 controls how toread data of individual pixels of the frame memory, equivalentlymeritorious effect can be materialized. For example, when generating adisplay image as shown at (c) in FIG. 2, one line of a picked up imagethe image pickup unit 0101 outputs is first read at intervals of aconstant number of pixels and subsequently, one line of data inside acutout area is read at intervals of a constant number of pixels. Then,this operation repeats itself by the number of lines of the liquidcrystal display. As will be seen from the above, the method ofgenerating a display image by controlling the manner of reading theframe memory is complicated in control but is advantageous in thatcutout image data need not be generated separately.

As described above, according to the present embodiment, in the imagingapparatus having the display monitor such as a liquid crystal display,the visual recognition capability can be improved and the operabilitywhen the focusing and exposure are conducted manually can also beimproved by displaying part of an image at ×1 magnification or at anenlargement magnification and besides, the user's convenience inoperation can be improved by switching the display method in accordancewith the operation by the user.

Embodiment 2

Referring to FIG. 8, an imaging apparatus according to a secondembodiment of the present invention is illustrated schematically. InFIG. 8, the imaging apparatus comprises an image pickup unit 0101, acutout image generation unit 0102, an image display unit 0103, a displayimage control unit 0104 and a subject recognition unit 0811.

In the imaging apparatus as shown in FIG. 8, the image pickup unit 0101,in which zoom and focus are variable, includes a group of lensesrepresenting a zoom lens and a focus lens, a shutter, an image-pickupdevice such as CCD or CMOS, an AGC, an AD, a camera signal processingDSP, a moving picture processing LSI and a timing generator and operatesto perform image pickup based on photoelectric conversion and deliverimage data and image data for display which is reduced in accordancewith the resolution of a display area of the image display unit 0103.The image data for display may be generated by thinning the originalimage data at intervals of a predetermined number of pixels or byperforming an image reduction process based on a filter operation usingthe moving picture processing LSI. The image data is delivered to animage recording unit, not shown, so that a moving picture and a stillpicture can be recorded on a recording medium such as DVD, BD or HDD oralternatively, can be outputted to a TV monitor or PC display, notshown, so as to be displayed thereon. The cutout image generation unit0102 selects a predetermined area of the image data delivered out of theimage pickup unit 0101 and processes an image within the area at ×1magnification or at a predetermined enlargement magnification so as todeliver the resultant image as cutout image data. The image display unit0103 is a monitor unit such as a liquid crystal display attached to theimaging apparatus and it responds to information concerning control ofan image display method outputted from the display image control unit0104 to acquire and display image data for display delivered out of theimage pickup unit 0101 or cutout image data delivered out of the cutoutimage generation unit 0102. The display image control unit 0104 acquiresfrom the subject recognition unit 0811 a result of subject recognition,determines an image display method of the image display unit 0103 on thebasis of the captured result and outputs information for control of theimage display method to the image display unit 0103. The subjectrecognition unit 0811 performs detection and chase of the subject on thebasis of image data delivered out of the image pickup unit 0101 throughan image recognition process and outputs a result of subject recognitionto the display image control unit 0104. Also, the subject recognitionunit 0811 determines the position and range of a cutout area by thecutout image generation unit 0102 on the basis of the result of subjectrecognition and outputs it to the cutout image generation unit 0102. Thesubject signifies herein a noticed objective such as a characteristicarea, for example, a face of a person or a moving body and for detectionof the subject, the existing face detection technique or the movingobject detection technique may be used. For chase of the subject, thesubject may be detected frame by frame and detection results aremutually correlated or alternatively, the existing chase techniquetypified by optical flow or inter-frame difference may be employed. Theimage recognition process the subject recognition unit 0811 conducts istypically executed by the microcomputer, camera signal processing DSP orthe dedicated LSI incorporated in the camera.

Illustrated in FIG. 9 is an example of a method of controlling a cutoutarea of a cutout image in the second embodiment of the invention. InFIG. 9, picked up images displayed on the liquid crystal display beforeand after a lapse of constant frame are shown at (a) and correspondingcutout images displayed on the liquid crystal display before and afterthe constant frame lapse are shown at (b). In the drawings, 0901_1designates a cutout area before the constant frame lapse, 0901_2 acutout area after the constant frame lapse, 0902_1 a subject personbefore the constant frame lapse and 0902_2 a subject person after theconstant frame lapse. Dotted line and arrow indicated at (a) in FIG. 9are displayed for convenience of explanation and are not subject imagesbeing actually picked up. Operation in a method for controlling a cutoutarea of a cutout image will be described by using FIG. 9. It is nowassumed that before the constant frame lapse, the user operates todisplay the area 0901_1 nearby the subject person 0902_1 at ×1magnification on the liquid crystal display. At that time, the subjectrecognition unit 0811 detects the subject person 0902_1 contained in thecutout area 09011_1. When the subject person has moved in arrowdirection in the drawing at the termination of the constant frame lapse,the subject recognition unit conducting a process of chasing the subjectperson on the basis of image recognition detects the subject person0902_2 after the movement and controllably causes the cutout area toshift on the basis of an amount of movement and a direction of movement.The control information is outputted to the cutout image generation unit0102 to cause it to change the cutout area on the basis of the controlinformation, so that the position of the subject can be so controlled asto remain throughout the movement as shown at (b) in FIG. 9. Thus, evenwhen the subject at which the user wants to set focus and exposuremoves, the area to be displayed at ×1 or enlargement magnification canbe chased automatically and accurate in-focus and exposure conditionscan be realized easily. In this example, the explanation is given of thesingle subject but in case a plurality of subjects are present, the usermay select a noticed subject with the help of a touch panel not shown,the subject recognition unit 0811 may detect factors such as positionsand area ratios of the individual subjects inside the cutout areas andmay weight the factors, respectively, in order that the noticed subjectcan be determined. Further, if a subject moves forwardly or backwardlyand the area of the subject in a cutout image changes, the magnificationof enlargement of the cutout image may be controlled on the basis of anarea of the subject to enable the subject to be displayed constantly insubstantially the same size. In conducting the subject chase using theimage recognition, the chase process may be executed only within acutout area when the moving speed of the subject is low, therebyreducing the process cost. If the moving speed of the subject is so highthat the subject departs from an angle of field of the cutout area, thechase process is carried out at the entire field angle for a normalpickup image in order that the subject being outside the field angle ofthe cutout area can be chased.

Illustrated in FIG. 10 is an example of a method of controlling an imagedisplay method in the second embodiment of the invention. In FIG. 10,picked up images displayed on the liquid crystal display before andafter a lapse of constant frame are shown at (a) and correspondingcutout images displayed on the liquid crystal display before and afterthe constant frame lapse are shown at (b). In the drawings, 1001_1designates a cutout area before the constant frame lapse, 1001_2 acutout area after the constant frame lapse, 1002_1 a first subjectperson before the constant frame lapse, 1002-2 the first subject personafter the constant frame lapse and 1003 a second subject person afterthe constant frame lapse. Dotted line indicated at (a) in FIG. 10 isdisplayed for convenience of explanation and is not a subject imagebeing actually picked up. Operation in a method for controlling an imagedisplay method will be described by using FIG. 10. It is now assumedthat before the constant frame lapse, the user operates to display thearea 1001_1 nearby the first subject person 1002_1 at ×1 magnificationon the liquid crystal display. The subject recognition unit 0811conducts the subject detection process even for an object outside thecutout area 1001_1 every frame or at intervals of constant frames. Whenthe subject recognition unit 0811 detects the second subject 1003outside the cutout are 1001_2 after the constant frame lapse, it outputsa detection result to the display image control unit 0104. Receiving thedetection result, the display image control unit 0104 controllablyswitches the display image to a normal pickup image as shown at (b) inFIG. 10. Thus, even when the user is operating by confirming only thecutout image, the screen can be switched automatically so as to permitthe entire field angle to be seen as the new subject comes into a fieldangle at which image pickup is being in progress and the subject can beconfirmed and the composition thereof can be grasped easily withoutmissing the subject coming into the field angle.

As described above, according to the present embodiment, in the imagingapparatus having the display monitor such as a liquid crystal display,the visual recognition capability can be improved and the operabilitywhen the focusing and exposure are conducted manually can be improved bydisplaying part of an image at ×1 magnification or at an enlargementmagnification and in addition, the user's convenience in operation canbe improved by changing the position and size of the area to bedisplayed at the enlargement magnification and by switching the imagedisplay method automatically through the image recognition.

Embodiment 3

Referring to FIG. 11, an imaging apparatus according to a thirdembodiment of the present invention is illustrated schematically. InFIG. 11, the imaging apparatus comprises a first image pickup unit1101_1, a second image pickup unit 1101_2, a cutout image generationunit 0102, an image display unit 0103, a display image control unit 0104and a stereo-image processing unit 1111.

In the imaging apparatus as shown in FIG. 11, each of the first andsecond image pickup units 1101_1 and 1101_2 in which zoom and focus arevariable, includes a group of lenses representing a zoom lens and afocus lens, a shutter, an image-pickup device such as CCD or CMOS, anAGC, an AD, a camera signal processing DSP, a moving picture processingLSI and a timing generator and operates to perform image pickup based onphotoelectric conversion and deliver image data. The stereo-imageprocessing unit 1111 conducts a stereo-image process by using image datadelivered out of each of the first and second image pickup units 1101_1and 1101_2 to perform three-dimensional measurement of a subject anddelivers three-dimensional information including parallax information,distance information, three-dimensional position information andevaluation values indicative of reliability of the three-dimensionalmeasurement. The stereo-image processing unit also reduces image datarepresenting that from one of the first and second image pickup units1101_1 and 1101_2 or synthetic image data of both data of the first andsecond image pickup units in accordance with the resolution of a displayarea of the image display unit 0103 and outputs the reduced data asstereo-image data for display. The stereo-image data for display may begenerated by thinning the original image data at intervals ofpredetermined pixels or by performing an image reduction process basedon a filter operation using the moving picture processing LSI. Thestereo-image process includes a pre-process typified by a calibrationprocess such as lens distortion correction process and parallelismprocess and a low-pass filter process for noise elimination as well, apost-process typified by a process for operating a feature quantity, forexample, edge detection, a stereo-matching process for making thecorrespondence between stereo-images by using various correlationprocesses inclusive of normalized mutual correlation and block matching,for example, absolute value sum and a space-sweep process so as toacquire parallax information and a process for removing a singular pointthrough rank filter process and labeling as well and besides, a distancecalculation process for operating distance information by using theparallax information, thereby ensuring that information obtained on theway of process, for example, the result of correspondence when thestereo-matching process is carried out can be outputted. The cutoutimage generation unit 0102 selects a predetermined area of image datadelivered out of either the first image pickup unit 1101_1 or secondimage pickup unit 1101_2 and delivers, as cutout image data, an imageinside the area at ×1 magnification or at an enlargement magnification.The image cutout area may be inputted from an interface not shown or maybe determined on the basis of various operation results delivered out ofthe stereo-image processing unit 1111. An example of a method ofdetermining the cutout area will be described later with reference toFIG. 12. Alternatively, a first cutout image generated from the imagedata delivered out of the first image pickup unit 1101_1 may besynthesized with a second cutout image generated from the image datadelivered out of the second image pickup unit 1101_2 to produce astereo-cutout image which in turn is outputted. The image display unit0103 is a monitor unit such as a liquid crystal display attached to theimaging apparatus and it responds to information concerning control of aimage display method outputted from the display image control unit 0104to acquire and display image data for display delivered out of thestereo-image processing unit 1111 or cutout image data or stereo-cutoutimage data delivered out of the cutout image generation unit 0102. Thedisplay image control unit 0104 acquires input information from aninterface, not shown, and various operation results delivered out of thestereo-image processing unit 1111 and determines an image display methodof the image display unit 0103 on the basis of the acquired informationand results and outputs information for controlling the image displaymethod to the image display unit 0103. The stereo-image process thestereo-image processing unit 1111 conducts is typically executed by themicrocomputer, DSP or image processing LSI incorporated in the camera.In the present construction, the two image-pickup units are used but thenumber of the image pickup units is not limitative and three or moreplural image pickup units located at different positions mayalternatively be employed. In addition, a stereo-image pickup unithaving a mirror and a prism may be employed, permitting a single imagepickup unit with a single image pickup device to be used.

Turning to FIG. 12, an example of an image display method in the thirdembodiment of the invention is illustrated. In FIG. 12, a normal imagepicked up by a left one of the two cameras located on right and left isdisplayed on the liquid crystal display as shown at (a), a normal imagepicked up by a right camera is displayed on the liquid crystal displayas shown at (b) and a superimposed image of cutout images by the rightand left cameras is displayed on the liquid crystal display as shown at(c). In the drawings, 1201_1 designates a cutout area in the pickupimage by the left camera, 1201_2 a cutout area in the pickup image bythe right camera, 1202_1 a subject person in the pickup image by theleft camera and 1202_2 a subject person in the pickup image by the rightcamera. Dotted lines indicated at (a) and (b) in FIG. 12 are displayedfor convenience of explanation and are not subject images being actuallypicked up. Operation in a method for controlling an image display methodwill be described by using FIG. 12. In the pickup image by the leftcamera as shown at (a) in FIG. 12, the cutout image generation unit 0102cuts out the area 1201_1 nearby the subject person 1202_1. Fordetermination of the image cutout area, setting values according to somemodes may be set and written in an EEPROM at the time of, for example,shipment to enable the user to input a selected value through an inputinterface, not shown, while watching the menu screen or an image cutoutarea may be set freely by the user through the use of a joint dial or atouch panel not shown. The stereo-image processing unit 1111 detects asubject person 1202_2 taken by the right camera corresponding to asubject person 1202_1 taken by the left camera through a stereo-matchingprocess and outputs a detection result to the cutout image generationunit 0102. In the stereo-matching process, the area 1201_1, for example,may be provided as a template and detection may proceed through matchingprocess on the area basis or feature points inside the area 1201_1 maybe detected so that points corresponding to the individual featurepoints can be detected through a matching process on the feature pointbasis and weighted results can be unified to determine a correspondingarea. The cutout image generation unit 0102 determines the cutout area1201_2 in the pickup image by the right camera in such a way that theposition of the subject person 1202_1 in the cutout image by the leftcamera substantially coincides with the position of the subject person1202_2 in the cutout image by the right camera. By synthesizing thecutout images taken by the left and right cameras, the cutout imagegeneration unit 0102 can display neighborhoods of subjects of therespective cameras at ×1 magnification or at an enlargementmagnification as shown at (c) in FIG. 12. In this example, thestereo-matching process is conducted with the pickup image by the rightcamera referenced to the left camera but conversely, it may be conductedwith the pickup image by the left camera referenced to the right camera.By making a juxtaposed display of neighborhoods of the correspondingsubjects by the left and right cameras at ×1 magnification or at anenlargement magnification, the user can be allowed to perform focusoperation and exposure operation while confirming conditions of focusand exposure for the pickup images by the left and right cameras at atime. Accordingly, even in case focus and exposure conditions for thesame subject delicately slightly differ between the cameras because of adifference in angle of field between the left and right cameras, thefocus positions can be unified accurately and besides the exposureconditions can also be unified accurately between the cameras to improvethe accuracies in stereo-matching process and three-dimensionalmeasurement in the stereo-image process.

As described above, according to the present embodiment, in the imagingapparatus having the display monitor such as a liquid crystal displayand the plurality of image pickup units, the neighborhoods of thesubjects corresponding to each other between the image pickup units canbe displayed at ×1 magnification or at an enlargement magnification andvisual recognition capability by the user can be improved during focusand exposure matching operation and therefore improvements in operationaccuracies in the stereo-image process can be assisted.

It should be understood that the present invention is not limited to theforegoing embodiments and can include various modifications andalterations. For example, the foregoing embodiments have been describedin detail for clear and better understanding of the invention and thisinvention is not always limited to include all constructions set forthso far. Further, part of construction of an embodiment can be replacedwith construction of another embodiment or the construction of anembodiment can be added to the construction of another embodiment.

The present invention has been described by way of preferred embodimentsof thereof.

According to this invention, in a video camera or still camera having adisplay monitor such as a liquid crystal display, an image on thedisplay monitor can be switched over through simplified operation orthrough automatic control to improve the convenience capability when theuser uses various functions while watching the display monitor.

The present invention can be utilized for a consumer use camera, amonitoring camera and a stereoscopic camera.

According to this invention, the user's convenience in operating theimaging apparatus can be improved. Especially, the convenience can beimproved for the user to use various functions while watching thedisplay monitor.

It should be further understood by those skilled in the art thatalthough the foregoing description has been made on embodiments of theinvention, the invention is not limited thereto and various changes andmodifications may be made without departing from the spirit of theinvention and the scope of the appended claims.

The invention claimed is:
 1. An imaging apparatus comprising: an imagerconfigured to perform image pickup and obtain a pickup image; an imagedisplay configured to display the pickup image; a touch panel configuredto enable a user to select a subject included in the pickup imagedisplayed on the image display; a controller configured to control theimage display to display the pickup image or a superimposed imageincluding the pickup image and a cutout image which is cut out andgenerated for displaying a part of an area of the pickup image withmagnification; an operation input interface configured to switch animage displayed on the image display from the pickup image to thesuperimposed image in accordance with an operation of a user; and arecording button configured to operate a start and an end of a recordingof the pickup image obtained by the imager when a moving picture isrecorded, wherein the controller is further configured to: when thesubject included in the pickup image displayed on the image display isselected via the touch panel, recognize and chase the selected subjectin the pickup image, in a case that a position of the selected subjectthat is recognized and chased in the pickup image is changed,automatically change the area which is cut out as the cutout image fromthe pickup image in accordance with the changed position of the subjectselected in the pickup image, and when the superimposed image includingthe pickup image and the cutout image including the selected subject isdisplayed on the image display in accordance with the operation via theoperation input interface and an operation of the start of the recordingfor recording the moving picture by the recording button is detected,switch from displaying the superimposed image to displaying the pickupimage on the image display.
 2. The imaging apparatus according to claim1, wherein the operation input interface is configured to receive amanual focus operation as the operation of the user, and wherein thecontroller is further configured to, when an input of the manual focusoperation is detected, control a focus position and switches fromdisplaying the pickup image to displaying the superimposed image on theimage display.
 3. The imaging apparatus according to claim 1, whereinthe controller is further configured to, when recording the pickup imageas the moving picture, display the pickup image on the image displaywithout switching to displaying the superimposed image during therecording when the operation is input via the operation input interface.4. The imaging apparatus according to claim 1, wherein the controller isfurther configured to operate in a control mode to recognize and chase aface of a person as the selected subject included in the pickup imagedisplayed on the image display, wherein the controller is furtherconfigured, in the control mode, to: in a case that a position of theface of the person that is recognized and chased in the pickup image ischanged, automatically change the area which is cut out from the pickupimage as the cutout image in accordance with the changed position of theface of the person in the pickup image, and when the superimposed imageincluding the pickup image and the cutout image including the face ofthe person is displayed on the image display in accordance with theoperation via the operation input interface and the operation of thestart of the recording for recording the moving picture by the recordingbutton is detected, switch from displaying the superimposed image todisplaying the pickup image on the image display.
 5. A method ofoperating an imaging apparatus comprising: picking up and obtaining apickup image; displaying, on an image display of the imaging apparatus,the pickup image; receiving, via a touch panel of the imaging apparatus,a selection of a subject included in the displayed pickup image; whenthe subject included in the pickup image displayed on the image displayis selected via the touch panel, recognizing and chasing the selectedsubject in the pickup image; switching, in accordance with an operationof a user via an operation input interface of the imaging apparatus,from displaying the pickup image to displaying a superimposed imageincluding the pickup image and a cutout image which is cut out andgenerated for displaying a part of an area of the pickup image withmagnification; and in a case that a position of the selected subjectthat is recognized and chased in the pickup image is changed,automatically changing the area which is cut out as the cutout imagefrom the pickup image in accordance with the changed position of theselected subject in the pickup image; when the superimposed imageincluding the pickup image and the cutout image including the selectedsubject is displayed on the image display in accordance with theoperation via the operation input interface and an operation of a startof recording of the pickup image for recording a moving picture is inputvia a recording button of the imaging apparatus, switching fromdisplaying the superimposed image to displaying the pickup image on theimage display.
 6. The method according to claim 5, further comprising:controlling, when an input of a manual focus operation is detected, afocus position of the pickup image, wherein the manual focus operationis the operation of the user input via the operation input interfacethat switches the display of the pickup image to the display of thesuperimposed image on the image display.
 7. The method according toclaim 5, further comprising: when recording the pickup image as themoving picture, displaying the pickup image on the image display withoutswitching to displaying the superimposed image during the recording whenthe operation is input via the operation input interface.
 8. The methodaccording to claim 5, further comprising: operating in a control mode torecognize and chase a face of a person as the selected subject includedin the pickup image displayed on the image display, wherein, in a casethat a position of the face of the person that is recognized and chasedin the pickup image is changed in the control mode, the area which iscut out from the pickup image as the cutout image is automaticallychanged in accordance with the changed position of the face of theperson in the pickup image, and wherein, in the control mode, when thesuperimposed image including the pickup image and the cutout imageincluding the selected subject is displayed on the image display inaccordance with the operation via the operation input interface and thestart of the recording by the recording button is detected, the displayof the superimposed image is switched to the display of the pickup imageon the image display.